Topic Shift Analysis & Validation

Overview

Now that we have labels for out topic we can plot the topic shift and review statistics that might help us understand trends across the different models.

These metrics describe the behavior and nature of each topic over time.

Topic Volatility

How “spiky” or “stable” is a topic’s prominence over time?

For each topic, calculate the standard deviation or coefficient of variation of its average gamma scores across all speeches.

Interpretation: A high volatility score indicates an “event-driven” topic that appears intensely and then fades (e.g., war, pandemic). A low score suggests a stable, persistent “background” topic that is always present.

Topic Persistence

Does a topic remain relevant long after its peak, or does it disappear completely? For each topic, calculate a “persistence score” by dividing its average gamma in the most recent year by its all-time peak gamma.

Interpretation: A score close to 1 means the topic is still highly relevant today, even if its peak was long ago. A score close to 0 means the topic has largely faded from the discourse.

Topic Exclusivity

How unique are the words that define a topic? For each topic, count how many of its top 15 words are not in the top 15 words of any other topic in the same model.

Interpretation: High exclusivity suggests a very distinct and well-defined theme. Low exclusivity suggests a more general or foundational topic whose vocabulary is shared across many different policy areas.

4 Topics

We use the 4 topic model for more defined and interpretable topics.

Topic Statistics and Cadence (Yearly)

	Topic	Peak Year	Volatility	Persistence	Exclusivity
	International Affairs & Defense	1878	0.45	0	17
	Parliamentary Procedure & Administration	1946	0.37	0	12
	Economic Development & Growth	1980	0.32	0	9
	Social Policy & Community Welfare	2013	0.34	1	16

The plot is striking. The topics are well defined and a clear pattern of a synchronized raising and declining prominence (stable Volatility). The topic labels seem to fit well with the eras each topic represents.

8 Topics

The 8 topics model is more temporal. Even though there more topics they are evenly exclusive.

Topic Statistics and Cadence (Yearly)

	Topic	Peak Year	Volatility	Persistence	Exclusivity
	Federal-Provincial Coordination	1910	0.45	0.00	12
	Parliamentary Business & Trade	1931	0.30	0.00	9
	War & International Relations	1944	0.28	0.00	10
	Jobs & Economic Policy	1968	0.20	0.00	5
	Infrastructure & Regional Development	1976	0.21	0.00	8
	Economic Development Programs	1983	0.23	0.40	6
	Indigenous Affairs & Climate	2013	0.23	0.40	7
	Social Programs & Healthcare	2020	0.24	0.21	8

We now see topic that declined in prominence and later reemerge, like “War & International Relations”. We see periods where topics peek with less prominence. Topics in the early and late years, appear to have strong prominence.

Validation against Major Events

To validate the topics prominence we compare them against major events and governments.

Major Canadian Historical Events

The table below lists partial but uncontroversial, major Canadian historical events marked in the plots below for.

Major Canadian Historical Events Since 1867

Date	Event
1867	Confederation: The Dominion of Canada is formed.
1885	Completion of the Canadian Pacific Railway.
1914	Canada enters World War I.
1918	World War I ends.
1929	Great Depression begins.
1939	Canada enters World War II.
1945	World War II ends.
1959	Suez Crisis
1973	Oil Embargo
1982	Canada Act 1982: Patriation of the Constitution.
1995	Widespread adoption of the internet.
2001	Canada joins the Afghanistan War.
2020	COVID-19 pandemic: WHO declares a global pandemic.

Governments

We can also compare the topics prominence against the different governments and see if they “fit”.

Conclusion

This topic modeling analysis of Canadian Speeches from the Throne reveals meaningful patterns in political discourse that align closely with historical events and governmental transitions. The 4-topic model demonstrates clear temporal succession, with Parliamentary Procedure & Administration dominating the early confederation period, followed by International Affairs & Defense during the world wars, Economic Development & Growth in the post-war boom, and Social Policy & Community Welfare in the modern era. The 8-topic model provides additional granularity, capturing the cyclical nature of themes like “War & International Relations” that resurge during different conflicts, and revealing how contemporary issues like Indigenous Affairs & Climate have gained prominence in recent decades. The validation against major historical events—from Confederation through both World Wars, the Great Depression, and the COVID-19 pandemic—confirms that these computational methods successfully identify genuine shifts in political priorities.

These findings demonstrate that unsupervised machine learning can effectively uncover the evolution of political discourse, providing quantitative evidence for how Canadian political priorities have shifted across nearly 160 years of parliamentary debate. The use of Speeches from the Throne as our corpus was particularly valuable, as these ceremonial addresses represent a standardized format delivered at consistent intervals, creating a reliable yardstick for measuring thematic changes over time. Unlike other parliamentary speeches that vary widely in context and purpose, Throne Speeches serve as an index of governmental priorities, allowing us to track shifts in political focus with greater precision and comparability across different eras and administrations.